* fix flux controlnet pipelines

---------

Co-authored-by: yiyixuxu <[email protected]>

* refactor scheduler class usage

* reorder to make tests more readable

* remove pipeline specific checks and skip tests directly

* rewrite denoiser conditions cleaner

* bump tolerance for cog test

* check if default attn procs fix it.

* print

* print

* replace

* style./

* replace revision with variant.

* replace with stable-diffusion-v1-5/stable-diffusion-inpainting.

* replace with stable-diffusion-v1-5/stable-diffusion-v1-5.

* fix

* fix positional arguments in check_inputs().

* add video and latetns to check_inputs().

* prep latents_in_channels.

* quality

* multiple fixes.

* fix

* Reduced model size for lumina-tests

* Handled failing tests

…ing. (#9498)

Fix the bug of sd3 controlnet training when using gradient_checkpointing. Refer to issue #9496

* exponential sigmas

* Apply suggestions from code review

Co-authored-by: YiYi Xu <[email protected]>

* make style

---------

Co-authored-by: YiYi Xu <[email protected]>

Co-authored-by: YiYi Xu <[email protected]>

…#9504)

* Add Noise Schedule/Schedule Type to Schedulers Overview docs

* Update docs/source/en/api/schedulers/overview.md

Co-authored-by: Steven Liu <[email protected]>

---------

Co-authored-by: Steven Liu <[email protected]>

Co-authored-by: Sayak Paul <[email protected]>
Co-authored-by: YiYi Xu <[email protected]>

* Fix bug

* import imageio

* allow faster downloads from the Hub in CI.

* HF_HUB_ENABLE_HF_TRANSFER: 1

* empty

* empty

* remove ENV HF_HUB_ENABLE_HF_TRANSFER=1.

* empty

* update sd15 repo

* update more

* batched implementation of flux cfg.

* style.

* readme

* remove comments.

* flux shard

* feedback

* up

* Update src/diffusers/models/modeling_utils.py

Co-authored-by: Aryan <[email protected]>

---------

Co-authored-by: Aryan <[email protected]>

Fixed latents_to_rgb doc.

Co-authored-by: Álvaro Somoza <[email protected]>

Add beta sigmas / beta noise schedule

* flux controlnet mode to take into account batch size

* incorporate yiyixuxu's suggestions (cleaner logic) as well as clean up control mode handling for multi case

* fix

* fix use_guidance when controlnet is a multi and does not have config

---------

Co-authored-by: Christopher Beckham <[email protected]>
Co-authored-by: Sayak Paul <[email protected]>

fix

)

* bugfix: precedence of operations should be slicing -> tiling

* fix typo

* fix another typo

* deprecate current implementation of tiled_encode and use new impl

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

Co-authored-by: YiYi Xu <[email protected]>

* Update src/diffusers/models/autoencoders/autoencoder_kl.py

---------

Co-authored-by: Sayak Paul <[email protected]>
Co-authored-by: YiYi Xu <[email protected]>

* make set_adapters() method more robust.

* remove patch

* better and concise code.

* Update src/diffusers/loaders/lora_base.py

Co-authored-by: YiYi Xu <[email protected]>

---------

Co-authored-by: YiYi Xu <[email protected]>

* add train flux-controlnet scripts in example.

* fix error

* fix subfolder error

* fix preprocess error

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <[email protected]>

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <[email protected]>

* fix readme

* fix note error

* add some Tutorial for deepspeed

* fix some Format Error

* add dataset_path example

* remove print, add guidance_scale CLI, readable apply

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <[email protected]>

* update,push_to_hub,save_weight_dtype,static method,clear_objs_and_retain_memory,report_to=wandb

* add push to hub in readme

* apply weighting schemes

* add note

* Update examples/controlnet/README_flux.md

Co-authored-by: Sayak Paul <[email protected]>

* make code style and quality

* fix some unnoticed error

* make code style and quality

* add example controlnet in readme

* add test controlnet

* rm Remove duplicate notes

* Fix formatting errors

* add new control image

* add model cpu offload

* update help for adafactor

* make quality & style

* make quality and style

* rename flux_controlnet_model_name_or_path

* fix back src/diffusers/pipelines/flux/pipeline_flux_controlnet.py

* fix dtype error by pre calculate text emb

* rm image save

* quality fix

* fix test

* fix tiny flux train error

* change report to to tensorboard

* fix save name error when test

* Fix shrinking errors

---------

Co-authored-by: YiYi Xu <[email protected]>
Co-authored-by: Sayak Paul <[email protected]>
Co-authored-by: Your Name <[email protected]>

* clean up the serialization stuff.

* better

* fix variant-idenitification.

* fix variant

* fix sharded variant checkpoint loading.

* Apply suggestions from code review

* fixes.

* more fixes.

* remove print.

* fixes

* fixes

* comments

* fixes

* apply suggestions.

* hub_utils.py

* fix test

* updates

* fixes

* fixes

* Apply suggestions from code review

Co-authored-by: YiYi Xu <[email protected]>

* updates.

* removep patch file.

---------

Co-authored-by: YiYi Xu <[email protected]>

* remove conv cache from the layer and pass as arg instead

* make style

* yiyi's cleaner implementation

Co-Authored-By: YiYi Xu <[email protected]>

* sayak's compiled implementation

Co-Authored-By: Sayak Paul <[email protected]>

---------

Co-authored-by: Sayak Paul <[email protected]>
Co-authored-by: YiYi Xu <[email protected]>

…hs` is passed in a distributed training env (#9316)

Fixed pix2pix lr scheduler

Co-authored-by: Sayak Paul <[email protected]>

* fix: retain memory utility.

* fix

* quality

* free_memory.

* support kohya flux loras that have tes.

* Add pag to controlnet inpainting pipeline


---------

Co-authored-by: YiYi Xu <[email protected]>

* Support bfloat16 for Upsample2D

* Add test and use is_torch_version

* Resolve comments and add decorator

* Simplify require_torch_version_greater_equal decorator

* Run make style

---------

Co-authored-by: Sayak Paul <[email protected]>
Co-authored-by: YiYi Xu <[email protected]>

Co-authored-by: Aryan <[email protected]>